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Abstract. This paper tackles three algorithmic problems for probabilis¬ 
tic automata on finite words: the Emptiness Problem, the Isolation Prob¬ 
lem and the Value 1 Problem. The Emptiness Problem asks, given some 
probability 0 < A < 1, whether there exists a word accepted with prob¬ 
ability greater than A, and the Isolation Problem asks whether there 
exist words whose acceptance probability is arbitrarily close to A. Both 
these problems are known to be undecidable [8,2,3]. About the Empti¬ 
ness problem, we provide a new simple undecidability proof and prove 
that it is decidable for automata with one probabilistic transition and 
undecidable for automata with as few as two probabilistic transitions. 
The Value 1 Problem is the special case of the Isolation Problem when 
A = 1 or A = 0. The decidability of the Value 1 Problem was an open 
question. We show that the Value 1 Problem is undecidable. Moreover, 
we introduce a new class of probabilistic automata, (t- acyclic automata, 
for which the Value 1 Problem is decidable. 


Introduction 

Probabilistic automata on finite words are a computation model introduced by 
Rabin [10]. Like deterministic automata on finite words, a probabilistic automa¬ 
ton reads finite words from a finite alphabet A. Each time a new letter a € A is 
read, a transition from the current state s £ Q to a new state t € Q occurs. In a 
deterministic automaton, t is a function of s and a. In a probabilistic automaton, 
a lottery determines the new state, according to transition probabilities which 
depend on the current state s and letter a. 

Since the seminal paper of Rabin, probabilistic automata on finite words 
have been extensively studied, see [4] for a survey of 416 papers and books 
about probabilistic automata published in the 60s and 70s. 

Quite surprisingly, relatively few algorithmic results are known about prob¬ 
abilistic automata on finite words and almost all of them are undecidability 
results. There are two main algorithmic problems for probabilistic automata on 
finite words: the Emptiness Problem and the Isolation Problem. The Empti¬ 
ness Problem asks, given some probability 0 < A < 1, whether there exists 
a word accepted with probability greater than A, while the Isolation Problem 



asks whether there exist words whose acceptance probability is arbitrarily close 
to A. Both these problems were shown undecidable, respectively by Paz [8] and 
Bertoni [2,3]. To our knowledge, the known decidability results known for proba¬ 
bilistic automata on finite words are rather straightforward: they either apply to 
one-letter probabilistic automata, in other words Markov chains, or to problems 
where the probabilistic nature of the automaton is not taken into account. 

In contrast, several algorithmic results recently appeared for probabilistic 
automata on infinite words. In [1], Baier et al. proved that for probabilistic Biichi 
automata, the emptiness problem may be undecidable (when the acceptance 
condition is ’’positive Biichi”) or decidable (for ”almost-sure Biichi”). In [9] is 
presented a class of probabilistic Biichi automata which recognize exactly u>- 
regular languages. In this paper, we only consider automata on finite words but 
several of our results seem to be extendable to probabilistic automata on infinite 
words. 

Our contributions are the following. 

First, we provide in Section 2.1 a new proof for the undecidability of the 
Emptiness Problem. 

Second, we strengthen Paz result: the Emptiness Problem is undecidable even 
for automata with as few as two probabilistic transitions (Proposition 4) and is 
decidable for automata with one probabilistic transition (Proposition 3). 

Third, we solve an open problem: Bertoni’s result shows that for any fixed 
cut-point 0 < A < 1, the Isolation Problem is undecidable. However, as stated 
by Bertoni himself, the proof seems hardly adaptable to the symmetric cases 
A = 0 and A = 1. We show that both these cases are undecidable as well, in 
other words the Value 1 Problem is undecidable (Theorem 4). 

Fourth, we introduce a new class of probabilistic automata, jj- acyclic au¬ 
tomata. , for which the Value 1 Problem is decidable (Theorem 5). To our knowl¬ 
edge, this is the first non-immediate decidability result for probabilistic automata 
on finite words. Moreover, we believe this is a first interesting step towards the 
design of classes of stochastic games with partial observation for which the value 
1 problem is decidable. 

These undecidability results show once again that probabilistic automata are 
very different from deterministic and non-deterministic automata on finite of in¬ 
finite words, for which many algorithmic problems are known to be decidable 
(e.g. emptiness, universality, equivalence). Surprisingly maybe, we remark that 
several natural decision problems about deterministic and non-deterministic au¬ 
tomata are undecidable as well (Corollaries 1 and 2). 

1 Probabilistic Automata 

A probability distribution on Q is a mapping S £ [0,1]^ such that X^seS $( s ) = 1- 
The set (s G Q \ 6(s) > 0} is called the support of S and denoted Supp(6). For 
every non-empty subset S' C Q, we denote 6s the uniform distribution on S 
defined by S(q) = 0 if q £ S and S(q) = if q £ S. We denote V(Q) the set of 
probability distributions on Q. 



Formally, a probabilistic automaton is a tuple A = (Q,A,(M a ) ae A,Qo,F), 
where Q is a finite set of states, A is the finite input alphabet, (. M a ) ae A are 
the transition matrices, qo is the initial state and F is the set of accepting 
states. For each letter a £ A, M a £ [0,1] ( 3 X< ? defines transition probabilities: 
0 < M a (s,t) < 1 is the probability to go from state s to state t when reading 
letter a. Of course, for every s € S and a £ A, Y^,teS M a (s, t) = 1, in other word 
in the matrix M a , the line with index s is a probability distribution on Q. 

Transition matrices define a natural action of A* on T>{Q). For every word 
a £ A we denote 6 ■ a the probability distribution in V(Q) defined by (<5 • a)(t) = 
J2 sgq d( s )'M a ( s , t). This action extends naturally to words of A*: Vtu £ A*,Ma £ 
A, S ■ ( wa ) = (S ■ w) ■ a. 

The computation of A on an input word w = ao ... a n £ A* is the sequence 
(So, Si,. .., S n ) £ V(Q) n+1 of probability distributions over Q such that <5o = 
<5{ qo } and for 0 < i < n, 5i+\ = Si ■ a,i. 

For every state q £ Q and for every set of states R C Q, we denote Pa((? 

R) = J2 reR (S q • w)(r) the probability to reach the set R from state q when 
reading the word w. 

Definition 1 (Value and acceptance probability). The acceptance proba¬ 
bility of a word w £ A* by A is P^(rc) = P^(?o —> F). The value of A, denoted 
val(A), is the supremum acceptance probability: val(A) = sup wgj 4 , P^(rc). 

2 The Emptiness Problem 

Rabin defined the language recognized by a probabilistic automaton as Ca( A) = 
{w £ A* | P^(ru) > A}, where 0 < A < 1 is called the cut-point. Hence, a 
canonical decision problem for probabilistic automata is: 

Problem 1 (Emptiness Problem) Given a probabilistic automaton A and 
0 < A < 1, decide whether there exists a word w such that P^('U') > A. 

The Strict Emptiness Problem is defined the same way except the large in¬ 
equality ¥ a (w) > \ is replaced by a strict inequality P^('U') > 

The special cases where A = 0 and A = 1 provide a link between proba¬ 
bilistic and non-deterministic automata on finite words. First, the Strict Empti¬ 
ness Problem for A = 0 reduces to the emptiness problem of non-deterministic 
automata, which is decidable in non-deterministic logarithmic space. Second, 
the Emptiness Problem for A = 1 reduces to the universality problem for non- 
deterministic automata, which is PSPACE-complete [6]. The two other cases are 
trivial: the answer to the Emptiness Problem for A = 0 is always yes and the 
answer to the Strict Emptiness Problem for A = 1 is always no. 

In the case where 0 < A < 1, both the Emptiness and the Strict Emptiness 
Problems are undecidable, this was proved by Paz [8]. The proof of Paz is a re¬ 
duction of the Emptiness Problem to an undecidable problem about free context 
grammars. An alternative proof was given by Madani, Hanks and Condon [7], 
based on a reduction to the emptiness problem for two counter machines. Since 



Paz was focusing on expressiveness aspects of probabilistic automata rather than 
on algorithmic questions, his undecidability proof is spread on the whole book [8], 
which makes it arguably hard to read. The proof of Madani et al. is easier to 
read but quite long and technical. 

In the next section, we present a new simple undecidability proof of the 
Emptiness Problem. 


2.1 New proof of undecidability 

In this section we show the undecidability of the (Strict) Emptiness Problem for 
the cut-point ^ and for a restricted class of probabilistic automata called simple 
probabilistic automata: 

Definition 2 (Simple automata). A probabilistic automaton is called simple 
if every transition probability is in {0, l}. 

The proof is based on a result of Bertoni [2]: the undecidability of the Equality 
Problem. 

Problem 2 (Equality problem) Given a simple probabilistic automaton A, 
decide whether there exists a word w € A* such that P^(u;) = 

Proposition 1 (Bertoni). The equality problem is undecidable. 

The short and elegant proof of Bertoni is a reduction of the Post Correspon¬ 
dence Problem (PCP) to the Equality Problem. 

Problem 3 (PCP) Let (p\ : A — >- {0,1}* and tp 2 '■ A — > {0,1}* two functions, 
naturally extended to A*. Is there a word w £ A* such that ? 

Roughly speaking, the proof of Proposition 1 consists in encoding the equality 
of two words in the decimals of transition probabilities of a well-chosen prob¬ 
abilistic automaton. While the reduction of PCP to the Equality problem is 
relatively well-known, it may be less known that there exists a simple reduction 
of the Equality problem to the Emptiness and Strict Emptiness problems: 

Proposition 2. Given a simple probabilistic automaton A, one can compute 
probabilistic automata B and C whose transition probabilities are multiple of j 
and such that: 

€ A+^^w) = ^ ^3u> e A + ,P b (w) > (1) 

<*=► (3«ed+,PcW>iJ . (2) 

Proof. The construction of B such that (1) holds is based on a very simple fact: 
a real number x is equal to ^ if and only if x(l — x) > Consider the automaton 
B which is the cartesian product of A with a copy of A whose accepting states 



are the non accepting states of A. Then for every word w £ A*, P^ 1 ('u;) = 
P^(w)(l — P^(w)), thus (1) holds. 

The construction of C such that (2) holds is based on the following idea. 
Since A is simple, transition probabilities of B are multiples of A, thus for every 
word w of length |iu|, Pg(w;) is a multiple of . As a consequence, Pb(w) > \ 
if and only if Pg(w) > \ — ■ Adding three states to B, one obtains easily 

a probabilistic automaton C such that for every non-empty word w £ A* and 
letter a £ A, P c(aw) = \ • Pg(ic) + \ ■ thus (2) holds. To build C, simply 
add a new initial state that goes with equal probability A either to the initial 
state of B or to a new accepting state qf. From qf, whatever letter is read, next 
state is qf with probability j and with probability | it is a new non-accepting 
absorbing sink state q*. □ 

As a consequence: 

Theorem 1 (Paz). The emptiness and the strict emptiness problems are un- 
decidable for probabilistic automata. These problems are undecidable even for 
simple probabilistic automata and cut-point A = A. 

To conclude this section, we present another connection between probabilistic 
and non-probabilistic automata on finite words. 

Corollary 1. The following problem is undecidable. Given a non-deterministic 
automaton on finite words, does there exist a word such that at least half of the 
computations on this word are accepting? 

At first, there is no obvious connection between this automata-theoretic deci¬ 
sion problem and probabilities. However we do not know a simple undecidability 
proof for this problem that does not make use of probabilistic automata. 

2.2 Probabilistic automata with few probabilistic transitions 

Hirvensalo showed that the emptiness problem is undecidable for probabilistic 
automata which have as few as 2 input letters and 25 states [5] . 

On the other hand, the emptiness problem is decidable for deterministic 
automata. This holds whatever be the number of states, as long as there are no 
probabilistic transition in the automaton. Formally, a probabilistic transition is 
a couple (s, a) of a state s £ S and a letter a £ A such that for at least one state 
t £ S, 0 < M a (s, t) < 1. 

This motivates the following question: what is the minimal number of prob¬ 
abilistic transitions for which the emptiness problem is undecidable? 

The answer to this question is somewhat surprising: the emptiness problem 
is decidable for automaton with one probabilistic transition (Proposition 3) but 
undecidable for automaton with as few as two probabilistic transitions (Propo¬ 
sition 5). 

Proposition 3. The emptiness problem for automata with one probabilistic tran¬ 
sition is decidable and PSPACE -complete. It is decidable in PTIME for au¬ 
tomata with bounded outdegree. 



A slight variant of the emptiness problem for probabilistic automata with 
one probabilistic transition is undecidable: 

Proposition 4. The following problem is undecidable: given a simple probabilis¬ 
tic automaton over an alphabet A with one probabilistic transition and given a 
rational language of finite words L C A*, decide whether P_ 4 (w) > \ for some 
word vu £ L. 

A simple corollary of Proposition 4 is the following undecidability result: 

Proposition 5. The emptiness problem is undecidable for probabilistic automata 
with two probabilistic transitions. 

Thus, the undecidability border is easily crossed when working with proba¬ 
bilistic automata. 

3 The Value 1 Problem 

In this section like in the previous one we give yet another way to cross the 
undecidability border, this time we follow the road of the value 1 -problem. 

3.1 Undecidability of the Value 1 problem 

In his seminal paper about probabilistic automata [10], Rabin introduced the 
notion of isolated cut-points. 

Definition 3. A real number 0 < A < 1 is an isolated cut-point with respect to 
a probabilistic automaton A if: 

3e > 0, \/w £ A*, |P_ 4 (w) — A| > e . 

Rabin motivates the introduction of this notion by the following theorem: 

Theorem 2 (Rabin). Let A a probabilistic automaton and 0 < A < 1 a cut- 
point. If A is isolated then the language £.4 (A) = {u £ A* \ P^(u) > A} is 
rational. 

This result suggests the following decision problem. 

Problem 4 (Isolation Problem) Given a probabilistic automaton A and a 
cut-point 0 < A < 1, decide whether A is isolated with respect to A. 

Bertoni [2] proved that the Isolation Problem is undecidable in general: 

Theorem 3 (Bertoni). The Isolation Problem is undecidable for probabilistic 
automata with five states. 

A closer look at the proof of Bertoni shows that the Isolation Problem is 
undecidable for a fixed A, provided that 0 < A < 1. 

However the same proof does not seem to be extendable to the cases A = 0 
and A = 1. This was pointed out by Bertoni in the conclusion of [2]: 



”Is the following problem solvable: 35 > 0,Vx,(p(x) > 5)? For au¬ 
tomata with 1-symbol alphabet, there is a decision algorithm bound with 
the concept of transient state [8]. We believe it might be extended but 
have no proof for it”. 

The open question mentioned by Bertoni is the Isolation Problem for A = 0. The 
case A = 1 is essentially the same, since 0 is isolated in an automaton A if and 
only if 1 is isolated in the automaton obtained from A by turning final states to 
non-final states and vice-versa. When A = 1, the Isolation Problem asks whether 
there exists some word accepted by the automaton with probability arbitrarily 
close to 1. We use the game-theoretic terminology and call this problem the 
Value 1 Problem. 

The open question of Bertoni can be rephrased as the decidability of the 
following problem: 

Problem 5 (Value 1 Problem) Given a probabilistic automaton A, decide 
whether A has value 1. 

U nfortunately, 

Theorem 4. The Value 1 Problem is undecidable. 

The proof of Theorem 4 is a reduction of the Strict Emptiness Problem to the 
Value 1 Problem. It is similar to the proof of undecidability of the Emptiness 
Problem for probabilistic Biichi automata of Baier et al. [1]. The core of the 
proof is the following proposition. 

Proposition 6. Let 0 < x < 1 and A x be the probabilistic automaton depicted 
on Fig. 1. Then A x has value 1 if and only if x > 

The proof of Theorem 4 relies on the fact that there is a natural way to 
combine A x with an arbitrary automaton B so that the resulting automaton 
has value 1 if and only if some word is accepted by B with probability strictly 
greater than t. 

The value 1 problem for simple probabilistic automata can be straigntfor- 
wardly rephrased as a ” quantitative” decision problem about non-deterministic 
automaton on finite words, which shows that: 

Corollary 2. This decision problem is undecidable: given a non-deterministic 
automaton on finite words, does there exists words such that the proportion of 
non-accepting computation pathes among all computation pathes is arbitrarily 
small? 


4 The class of (J-acyclic probabilistic automata 

In this section, we introduce a new class of probabilistic automata, acyclic 
probabilistic automata , for which the value 1 problem is decidable. 







a, b 



Fig. 1. This automaton has value 1 if and only if * > |. 


To get a decision algorithm for the value 1 problem, our starting point is the 
usual subset construction for non-deterministic automata, defined by mean of 
the natural action of letters on subsets of Q. However the quantitative aspect of 
the Value 1 Problem stressed in Corollary 2 suggests that the subset construction 
needs to be customized. Precisely, we use not only the usual action S ■ a of a 
letter a on a subset S C Q of states but consider also another action a®. Roughly 
speaking, a$ deletes states that are transient when reading letter a forever. 

Definition 4 (Actions of letters and ((-reachability) . Let A a probabilistic 
automaton with alphabet A and set of states Q. Given S C Q and a £ A, we 
denote: 

S ■ a = {t £ Q \ 3s £ S, M a (s,t) > 0} ■ 

A state t € Q is a-reachable from s € Q if for some n € N, Pyi(s -—>■ t) > 0. 
A state s € Q is a-recurrent if for any state t £ Q, 

(t is a-reachable from s) =>• (s is a-reachable from t) . 

A set S C Q is a-stable if S = S ■ a. If S is a-stable, we denote: 

S ■ = {s £ S | s is a-recurrent} . 

The support graph Qa of a probabilistic automaton A with alphabet A and 
set of states Q is the directed graph whose vertices are the non-empty subsets of 
Q and whose edges are the pairs ( S,T ) such that for some letter a £ A, either 
(S ■ a = T) or (S ■ a = S and S ■ = T). 

Reachability in the support graph of A is called jj -reachability in A. 

The class of ((-acyclic probabilistic automata is defined as follows. 

Definition 5 (((-acyclic probabilistic automata). A probabilistic automaton 
is ((-acyclic if the only cycles in its support graph are self-loops. 

Obviously, this acyclicity condition is quite strong. However, it does not for¬ 
bid the existence of cycles in the transition table, see for example the automaton 
depicted on Fig. 2. Note also that the class of ((-acyclic automata enjoys good 
properties: it is closed under cartesian product and parallel composition. 




Fig. 2. A {(-acyclic automaton (on the left) and its support graph (on the right). All 
transition probabilities are equal to \. 


4.1 The value 1 problem is decidable for ({-acyclic automata 

For (j-acyclic probabilistic automata, the value 1 problem is decidable: 

Theorem 5. Let A be a probabilistic automaton with initial state go and final 
states F. Suppose that A is (j -acyclic . Then A has value 1 if and only if F is 
{(-reachable from {go} in A. 

The support graph can be computed on the fly in polynomial space thus 
deciding whether a probabilistic automaton is j|-acyclic and whether an {(-acyclic 
automaton has value 1 are PSPACE decision problems. 

The rest of this section is dedicated to the proof of Theorem 5. This proof 
relies on the notion of limit-pathes. 

Definition 6 (Limit paths and limit-reachability). Let A be a probabilistic 
automaton with states Q and alphabet A. Given two subsets S, T of Q, we say that 
T is limit-reachable from S in A if there exists a sequence Wo, Wi, W 2 , ■ ■ ■ € A* 
of finite words such that for every state s € S: 

P^(s T) -» 1 . 

n—> oo 

The sequence Wo, Wi, W 2 , ■ ■ ■ is called a limit path from S to T, and T is said to 
be limit-reachable from S in A. 

In particular, an automaton has value 1 if and only if F is limit-reachable 
from {g 0 }- 

To prove Theorem 5, we show that in {(-acyclic automata, ^-reachability and 
limit-reachability coincide. The following proposition shows that ((-reachability 
always imply limit-reachability, may the automaton be {(-acyclic or not. 



Proposition 7. Let A be a probabilistic automaton with states Q and S, T C Q. 
IfT is fl- reachable from S in A then T is limit-reachable from S in A. 

The converse implication is not true in general. For example, consider the 
automaton depicted on Fig. 3. There is only one final state, state 3. The initial 
state is not represented, it leads with equal probability to states 1, 2 and 3. The 
transitions from states 1, 2 and 3 are either deterministic or have probability i. 


a 



b, b # 


Fig. 3. This automaton has value 1 and is not fl-acyclic . 


It turns out that the automaton on Fig. 3 has value 1, because ((6 n a)")„ e N 
is a limit-path from {1, 2, 3} to {3}. However, {3} is not reachable from {1, 2,3} 
in the support graph. Thus, limit-reachability does not imply ((-reachability in 
general. This automaton is not ((-acyclic , because his support graph contains a 
cycle of length 2 between {1, 2, 3} and {1,3}. It is quite tempting to add an edge 
labelled (ab^ between {1,3} and {3}. 

Now we prove that for ((-acyclic automata, limit-reachability implies ((-reachability. 

Definition 7 (Stability and ((-stability). Let A be a probabilistic automaton 
with states Q. The automaton A is stable if for every letter a £ A, Q is a-stable. 

A stable automaton A is ((-stable if for every letter a £ A Q ■ of = Q. 

The proof relies on the blowing, the flooding and the leaf lemmatas. 

Lemma 1 (Blowing lemma). Let A be a fl- acyclic probabilistic automaton 
with states Q and S C Q. Suppose that A is fl- acyclic and fl- stable. If Q is 
limit-reachable from S in A, then Q is fl- reachable from S as well. 



Proof (of the blowing lemma). If S = Q there is nothing to prove. If S ^ Q, 
we prove that there exists Si C Q such that (i) Si is jj-reachable from S, (ii) 
S C Si, and (iii) Q is limit-reachable from Si. Since S C Q and since there 
exists a limit-path from S to Q there exists at least one letter a such that S is 
not a-stable, i.e. S ■ a % S. Since A is subset-acyclic, there exists net) such 
that S • a n+1 = S • a n i.e. S ■ a n is a-stable. Let Si = (S • a") • af. Then (i) is 
obvious. 

To prove (ii), we prove that Si contains both S and S • a. Let s £ S. By 
definition, every state t of S • a" is a-accessible from s. Since A is ((-stable, 
state s is a-recurrent and by definition of a-recurrence, s is a-accessible from 
t. Since t £ S ■ a n and S • a” is a-stable, s £ S ■ a n and since s is a-recurrent 
s £ (S • a") - a** = Si. The proof that S • a C Si is similar. 

If Si = Q the proof is complete, because (i) holds. If Si C Q, then (iii) holds 
because S C Si thus Q is limit-reachable not only from S but from Si as well, 
using the same limit-path. As long as S„ ^ Q , we use (iii) to build inductively an 
increasing sequence S C Si C S 2 C . .. C S n = Q such that for every 1 < k < n, 
Sk+i is ((-reachable from Sfc. Since jj-reachability is transitive this completes the 
proof of the blowing lemma. □ 

Second, in a ((-stable and ((-acyclic automata, once a computation has flooded 
the whole state space, it cannot shrink back. 

Lemma 2 (Flooding lemma). Let A be a probabilistic automaton with states 
Q. Suppose that A is j (-acyclic and ((- stable. Then Q is the only set of states 
limit-reachable from Q in A. 

Now, we turn our attention to leaves of the acyclic support graph. 

Definition 8. Let A be a probabilistic automaton with states Q. A non-empty 
subset R C Q is called a leaf if for every letter a £ A, R ■ a = R and R- af = R. 

In a stable ((-acyclic automaton, there is a unique leaf: 

Lemma 3 (Leaf lemma). Let A be a probabilistic automaton with states Q. 
Suppose that A is ((-acyclic and stable. Then there exists a unique leaf fl- accessible 
from Q. Every set limit-reachable from Q contains this leaf. 

Proof (of the leaf lemma). Since A is ((-acyclic , there exists at least one leaf S 
which is ((-reachable from Q. 

We prove that every set limit-reachable from Q contains the leaf S. Let R 
limit-reachable from Q and (it n ) ra€ N a limit-path from Q to R. Since for every 
a £ A, S is a-stable, then a fortiori (u„) ra <=N is a limit-path from S to R(lS. Since 
S' is a leaf, the restriction A[S] of the automaton A to S is ((-stable. According to 
the flooding lemma applied to A[S], S = R.C\S, thus SCR. Since ((-reachability 
imply limit-reachability, this implies unicity of the leaf reachable from Q. □ 

To prove that limit-reachability implies ((-reachability, we proceed by induc¬ 
tion on the depth in the support graph. The inductive step is: 



Lemma 4 (Inductive step). Let A be a probabilistic automaton with states 
Q and Sq,T C Q. Suppose that A is jj -acyclic and T is limit-reachable from Sq. 
Then either So = T or there exists Si ^ Sq such that Si is jj- reachable from So 
in A and T is limit-reachable from Si in A. 

Repeated use of Lemma 4 gives: 

Proposition 8. Let A be a probabilistic automaton with states Q and Sq,T C 
Q. Suppose that A is jj- acyclic. If T is limit-reachable from So in A, then T is 
jj- reachable from So as well. 

Thus, limit-reachability and jj-reachability coincide in jj-acyclic automata and 
Theorem 5 holds. 

Is the maximal distance between two {(-reachable sets in the support graph 
bounded by a polynomial function of \A\ and |Q|? The answer to this question 
could lead to a simpler proof and/or algorithm. 

Conclusion 

The class of jj-acyclic automata can be probably extended to a larger class of 
automata for which the value 1 problem is still decidable. Another interesting 
research direction is to find classes of automata for which the emptiness problem 
is decidable, unfortunately this is not the case for jj-acyclic automata. 
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Appendix 

Proof (of Proposition 1). Given any instance , tp^ : A —> {0,1}* of the PCP 
problem, we build an automaton A which accepts some word with probability tj 
if and only if PCP has a solution. Let if : {0,1}* —> [0,1] the injective mapping 
defined by: 

; t \ a n , a 0 

ip{a 0 ...a n ) = — + ••• + — , 

and let 6 \ = ipoip 1 and 62 = 1(0 <fi 2 - Let Ai = ( Q , A, M, ql,q l F ) the probabilistic 
automaton with two states Q = {^g, q F } and transitions: 


Va £ A, M(a) 


l-6»i(a) 9i (a) 

1 - 61 (a) - 6 (a) + 


A simple computation shows that: 


Vu> £ A*, P Al (w) = 6 iW . (3) 

A very similar construction produces a two-states automaton A 2 such that: 

Vwed*, P_ 4 2 (w) = 1 - 62 (w) . (4) 


Let A be the disjoint union of these two automata A\ and A 2 plus a new initial 
state that leads with equal probability \ to one of the initial states q g and q% of 
Ai and A 2 • Then for every word w £ A* and every letter a € A, 


3w £ A*, P_ 4 (aiu) = 


3w£A*, ^P Al (w) 


ip^(^) = \ 


(3wgA*, 9 1 (w) = 6 2 (w)) 
(3tu € A*, tpi(w) = ip 2 (w)) 
PCP has a solution, 


where the first equivalence is by definition of A, the second is by (3) and (4), the 
third holds because ip is injective and the fourth is by definition of PCP. This 
completes the proof of Proposition 1. □ 

Proof. According to Proposition 1 and Proposition 2 the emptiness and the 
strict emptiness problems are undecidable for cut-point ^ and automata whose 
transition probabilities are multiples of i. The transformation of such automata 
into simple automata is easy. 


5 Proof of Proposition 3 

Proof. We first assume that the set of accepting states F is absorbant. Let 
(q, a) £ S x A be the unique couple of states and action such that 0 < M a (s, t) < 
1 for some states s, t. Without loss of generality, we can suppose that q is the 
initial state. Let n > 1 and {sg, • • • ,s„} be the set of states Si such that 0 < 



M a (q, Si). Let Ai be the deterministic automaton on finite words obtained from 
A by choosing Sj as initial state, removing the transition (q, a) and keeping the 
same final states. Denote C Si ^F Q A* the language accepted by A, and C Si ^ q 
the set of words labeling a path from s, to q for every 0 < i < n. 

We prove that : 


val a = max 

7xJC{s or . ,s„} 2 


Ma(<L s i) 1 
1 — AIa{q, Sj) j 


under the constraint: 



( 5 ) 


We first prove the direct inequality. Let /, </ C {s 0 ,--- ,s n } for wich the 
maximum is reached in (5). Let x\ = ")2 s . eI M a (q, Si) and X 2 = M a (q, Sj). 

Let w £ (f| Sie />C Si ^F) n We have (<^o • aw)(q) = x 2 and [S 0 ■ 

aw)(F) = X\. Since F is absorbant reading this word n times leads the following 


distribution (So ■ (aw) n )(F ) = Yli=o x 1 ' x 2 - Hence val a is greater than the right 
hand-side of (5). 

We prove the converse inequality. Let w £ A* a word accepted by A with 
probability p. If w is the empty string. Since w is accepted then q is accepting and 
p = 1 . Assume now that w is different from the empty string, w = au for some u £ 
A*. Let I = {i \ u £ C Si ^F} and J = {j \ u £ C s ^f}, for i and j in {0, • • • , n}. 
Let Ki and K 2 a partition of J such that K\ = {j £ J \ u £ C Sj ^ q } and K 2 = 
{j £ J I u (jL Cs^q}- We have P .4 (w) = Y) ie i M a (q, Si)+J2 jeK2 M (q, Sj)-¥ A (w), 
thus: 


Pa(w) 


J2ieiAIa(q, Sj) 

1 ~ YljeK 2 M(q, Sj) 


Since we are maximizing this quantity, it is clear that P_ 4 (u>) is less than the 
right hand-side of (5). 

If the set of accepting states is not absorbant. We construct a new automaton 
Ajj so that the value of val a = val a, . We add a new letter jj and a new absorbant 
state /# to A such that M$(s, s) = 1 if s is not accepting in A (j) has no effect on 
the automaton) and Mj(s, /#) = 1 if s is accepting in A. The unique accepting 
state in A$ is /#. It is easy to see that val a = val a, once we notice that for every 
word w £ A* accepted by A with probability p, the word wjj is accepted by A$ 
with same probability. 

To prove the PSPACE completeness we reduce the problem of deciding the 
intersection of n Automata on finite words which is known to be PSPACE com¬ 
plete [ 6 ]. Given n deterministic automata denoted A % = (Qi, A, Si, qfj, Ft) and 
i £ {1, • • • ,?r}, we construct the probabilistic automaton B by adding a new 
state qo and a new letter jj to the alphabet such that M$(q 0 , qfi) = The letter 



0 has no effect on the other states. If this probabilistic automaton has the value 
one, there exists then a word w £ A* such that for every i £ {l,-- - , n} we 
have w £ C Ai • Otherwise the intersection of all the Ai is empty. This shows the 
PSPACE completeness. □ 

6 Proof of Proposition 4 

Proof. To prove Proposition 4, we provide an algorithm which takes as input a 
simple probabilistic automaton A on an alphabet A (with an arbitrary number 
of probabilistic transitions) and outputs a simple automaton A' over an alphabet 
A! with only one probabilistic transition and a rational language L' C ( A ')* such 
that: 

(3w£A*,P A (w)>^j ^ (3w£L',F A ,(w')>^j . (6) 

According to Theorem 1, the Emptiness Problem for simple probabilistic au¬ 
tomata and cut-point ^ is undecidable, hence (6) proves Proposition 4. 

The intuition for the construction of A' ,L' and A! is as follows. In the sim¬ 
ple probabilistic automaton A, after a word u has been read, the probability 
distribution on states is 6 qo ■ u. Then, when a letter a is read, for every state 
p a coin toss is made to choose which of the successors go and q\ of p should 
be the next state (with qo = q\ if the transition (p,a) is deterministic). Then, 
the probability (S qo • u)(p) is split in twice ^( 5 qo ■ u)(p), moved to qo and q\ and 
added to probabilities that result of other simultaneous coin tosses. 

To simulate this phenomenon with only one probabilistic state, we realize 
the coin tosses one by one, rather than simultaneously. The introduction of new 
letters in the alphabet allows the automaton to toss coins state by state, while 
freezing the other states. Hence, while the original automaton would realize 
|Q| simultaneous transitions, the new automaton realizes 3|Q| + 1 sequential 
transitions. 

When the coin toss on p is simulated, the probability in p is moved to a new 
state s*, then split by a coin toss and then moved to two new states So and si. 
For the result of the coin toss not to interfere with subsequent coin tosses over 
qo and gi, the result is then stored in two new copy states go,# and gyjj, until all 
coin tosses have been realized and the simulation round is over. At the end of 
the simulation round, a “flushing letter” a# sends the results stored in new copy 
states gjj £ Q# back to the original states q £ Q, and a new simulation round 
begins. 

To implement this new automaton, only one probabilistic transition is needed: 
the transition from s* to so and si. Other transitions are deterministic. 

The rational constraint L' checks that coin tosses are realized in the right 
order and that the flushing letter is played at the end of each simulation round. 

Now we give the formal description of the automaton. The alphabet A', 
the automaton A! and the rational language V are computed as folllows. If 
A = (Q, A, M, qo, F) then A 1 = (Q', A', M!, qo, F) is defined by: 



1. The alphabet A! is made of two new letters a* (the coin toss letter) and aj 
(the flushing letter) plus, for each letter a € A and state s £ Q, two new 
letters a(a, s ) and /3(a, s ) so that: 

A’ = {a*,aj}U (J (a(a, s), /3(a, s)} . 

aeA,s£Q 

2. The set of states of A 1 is obtained from Q by addition of three new states 

s*, SO) Si, plus a copy q$ of each state q £ Q so that: 

Q Q U U fs*, so, si} . 

3. The initial state qo and the set of final states F are left unchanged. 

4. The transitions of A! are as follows. For every letter a £ A and state s £ Q, 

the new letter a(a, s ) has no effect on states u ^ s (i.e. M Q ( o s ) (it) = it), while 
from state s the transition is deterministic to state s* i.e. _M a ( ajS )(s) = s*. 

5. The new letter a* has no effect on states u ^ s*, while from state s* this is 
the only probabilistic transition of A', defined by M a ,(s *) = ^so + \s\. 

6. For every letter a £ A and state s £ Q, the new letter /3(a, s) lias no effect 
on states u qL (so, Si}. Transitions on letter /3(a, q) from states so and si are 
deterministic and depend on M a (s). If the transition M a (s) is deterministic, 
i.e. if M a (s, r) = 1 for some state r then M^r a ^(so) = rjj and M / 3 ( a s )(si) = 
rjj. If the transition M a (s) is probabilistic i.e. if M a (s) = \r + \r' for some 
states r, r' then M /3(ajS )(s 0 ) = rj and M /3 ( 0 iS )(si) = rj. 

7. The new letter aj lias no effect on states that are not in Q#. For every state 
Qi € Qjj which is a copy of a state q £ Q, the letter ajj sends deterministically 
state gj to q. 

Now we define the rational language L'. Choose some enumeration {so, Si,..., s 
Q of states of A and for each letter a, define the word 

u a = a(a, s 0 )a*/3(a, s 0 )a(a, si)a*/3(a, Si) • • • a(a, s„)a*/3(a, s n )a$ £ A'* , 

and let L' = {u a , a £ A}*. 

Then (6) holds because for every word 000102 • • • £ A*, 

P^(a 0 aia 2 • • •) = P A'(u ao u ai u a2 ...) . 

Since automaton A! has only one probabilistic transition, this completes the 
proof of Proposition 4. □ 

7 Proof of Proposition 5 

Proof. The undecidable problem described in Proposition 4 reduces to the empti¬ 
ness problem for simple probabilistic automata with two probabilistic transitions: 
given A and L 1 add a new initial state to A and from this new initial state, pro¬ 
ceed with probability \ either to the original initial state of A to the initial state 
of a deterministic automaton that checks whether the input word is in L. This 
new automaton accepts a word with probability more than | if and only if the 
original automaton accepts a word with probability more than ^. □ 



8 Proof of Proposition 6 

Proof (of Proposition 6). We shall prove: 

(x > i) <*=>■ (Ve > 0,3w e A*,P Ax (w) > 1 - e) . (7) 

In order to prove this equivalence we notice that: P_ 4 X (1 —^ 3) = x n and 

P^ x (4 —^ 6 ) = (1 — a:) n . Let (rik)k£N an increasing sequence of integers. By 
reading the word w = a n °ba ni b... a ni b 1 we get: 

/c>0 

< P^(4 A 6 ) = (1 - x) ni + (1 - (1 - x) ni ){l - x) 712 + ... 

. k> 0 k> 0 

If x < \ then P. 4 X (1 3) < P. 4 X (4 6 ) therefore no word w can be accepted 

with arbitrarily high probability if x < ^ what proves the converse implication 
on (7). Assume that x > ^7 we exhibit an increasing sequence of integers (nk)keN 
such that for every e > 0 we have: 

£ n ' c = 00 

fc -° ( 8 ) 

< e 

k>0 

Let Cel and rik = ln x (^) + C, notice that J2k> o( x ) Uk = x ° ■ J2k> 0 i ~ 00 ■ 
the other hand we have: 

1 - * = X ln * (1 - x) 

ln(l — x) 

— In x 

There exists /3 > 1 such that: l—x = x&, hence ^ fe>0 (l — x ) nh = Y^k> 0 x ^ nk ■ 

So: J2k >0 x ^ nk = X ^ C Sfe>o a^ lnx ^ = x^ c J]fc>o W' Si nce this series converge, 
we sati&fy (8) by choosing a suitable constant. It is easy to see that a sequence of 
finite words (a n °ba ni b... a ni b)i e n is accepted with probability arbitrarily close 
to 1. □ 

9 Proof of Theorem 4 

Proof (of Theorem 4)■ Given a probabilistic automaton B with alphabet A such 
that a,b (jL B, we combine B and the automaton A x on Fig.l to obtain an 
automaton C which has value 1 if and only if there exists a word w such that 



P_a(u>) > \. The input alphabet of C is Au{6} plus a new letter #. C is computed 
as follows. First, the transitions in A x on letter a are deleted. Second, we make 
two copies At and Ai of the automaton B, such that the initial state of A 4 is 4 
and the initial state of Ai is 1. From states of A 4 and At other than the initial 
states, reading letter b leads to the sink state 6. Third, from a state s of A 4 the 
transition on the new letter (t is deterministic and leads to 5 if s is a final state 
and to 4 if s is not a final state. Fourth, from a state s of Ai the transition on 
the new letter jj is deterministic and leads to 1 if s is a final state and to 2 if s 
is not a final state. Fifth, the final states of C are 5 and 3. Sixth, states 0, 3,6, 5 
and 2 are absorbing for letters in A. 

Then suppose there exists w such that P^i(ru) > \ and let us show that C 
has value 1. Let e > 0 and let u e = bd l °ba ll ba l2 b ■ ■ ■ a lk be a word accepted by B 
with probability 1 — e. Then by construction of C, 

P C (b(w$yH(w$rb(w$) i2 b---(w$y k )>P A (u e ) > 1 -e, 
thus C has value 1. 

Now suppose that for every w € A*, P^te < i and let us show that C has not 

value 1. Let w' £ (Au{b, ft})*. Factorize w' in w' = uoVo$UiVi$UkVk ■ ■ • such that 

Ui £ b* and Vi £ A*. Then by construction of C and by hypothesis, P 'c(w') < 

P.4i (uoauiau 2 a ■ ■ ■ Uka) < val(Ai). Thus val(C) < val(yti) and according to 
$ 2 2 

Proposition 6, val(C) <1. □ 


10 Proof of Theorem 5 

Proof (of Proposition 7). Proposition 7 is a consequence of the two following 
facts. 

First, if there is an edge from S to T in the support graph of A , then T is 
limit reachable from S: let S, T C Q and a € A. It S ■ a = T, then the sequence 
constant equal to a is a limit path from S to T. If S • a = S and S ■ o!f = T then 
by definition of S ■ of, (a")„ e N is a limit path from S to T. 

Second, limit-reachability is a transitive relation: let So, Si, S 2 C Q such that 
Si is limit-reachable from So and S 2 is limit-reachable from Si. Let a 

limit-path from So to Si and (v n )neN a limit-path from Si to S 2 . Then (u n v n ) ne ^ 
is a limit-path from So to S 2 . □ 

Proof (of the Flooding lemma). Let (■u„) n6 N be a limit path from Q to some set 
of states T C Q. We shall prove that T = Q. 

First, we prove that for every letter a £ At, Q \ T is a-stable. Otherwise 
there would be a £ At and t £ T such that t is a-reachable from some state 
s £ Q \ T. Since A is jj-stable, s and t are both a-recurrent, and by definition 
of a-recurrence, since t is a-reachable from s, s would be a-reachable from t as 
well. But s £ Q \ T and t £ T, which contradicts the a-stability of T. 

Second, we prove that u n £ Alf for only finitely many n £ N. Since for every 
a £ At, Q \ T is a-stable, then during the computation Sq = So, Si,.. . , <5| Mll | on 



the word u n , Y^sgQ\t <5fc( s ) is constant. Thus, for every n G N, 

P^(s T) = £ (6q • u n )(s) = = 'V Tl > °' 

seQ\T seQ\T ^ 

Since (it n ) n6 N is a limit-path from Q to T, P^(s T) converges to 0 hence 
the inequality can hold only for hnitely many n G N. 

Now we show that there exists T\ C Q such that: 


(i) 2i ^ T, 

(ii) T is {-reachable from T\ in A, 

(iii) and T\ is limit-reachable from Q in A. 

Since any inhnite subsequence of a limit-path is a limit-path, and since we proved 
that u n G Ap for only finitely many n G N, we can assume w.l.o.g. that for every 
n G N, u n Ap. Thus for every n G N, there exists v n G A*, a n G A \ At and 
w n G Ap such that u n = v n a n w n . W.l.o.g. again, since A is finite and T>(Q) is 
compact, we can assume that (a„)„ e N is constant equal to a letter a G A \ At 
and that (Sq ■ u n )„ S N converges to a probability distribution 5 G T>(Q). 

The choice of T\ such that (i),(ii) and (iii) hold depends on Supp(<5) • a. 

If Supp(d) • a = T then we choose T\ = Supp(<5). Then (i) holds because 
a £ At, (ii) holds because T = T\ ■ a and (iii) holds because (u„) ra€ N is a 
limit-path from Q to Tf. 

If Supp(d) -a^=T then we choose T\ = Supp(<5) • a. Then (i) clearly holds and 
(iii) holds because (i>„a) rae N is a limit path from Q to T) in A. To prove that (ii) 
holds, consider the restriction A[T, At] of automaton A to states T and alphabet 
At- Then (tu ra )„ e N is a limit-path from T) to T in A[T, At]. Moreover, since A 
is jj-acyclic and {(-stable, A[T, At] also is. Thus, we can apply the blowing lemma 
to A[T, At] and T), which proves that T is {-reachable from T\ in A[T, At], thus 
in A as well. 

If T\ = Q, the proof is complete. Otherwise, as long as T n ^ Q, we use 
condition (iii) to build inductively a sequence T = To, T\, T 2 , ■ ■ ■ T n such that for 
every 0 < k < n, ^ T ^+1 (condition (i))and T is {-reachable from Tk +1 in A 
(condition (ii)). Since A is {-acyclic , T n = Q after at most 2^ inductive steps. 

Since {-reachability is transitive, this proves that T is {-reachable from Q. 
Since A is {-stable, the only set {-reachable from Q is Q thus T = Q, which 
completes the proof of the flooding lemma. □ 

Proof (of Lemma 4)- Let («„) n gN be a limit-path from So to T. Let Aq = {a G 
A | So • a = So}. For every n G N, let v n be the longuest prefix of u n in Aq. 
Since every infinite subsequence of a limit-path is a limit-path, and since T>(Q) is 
compact, we can suppose without loss of generality that (Ss 0 ■ v n ) n e N converges 
to some distribution S G P(Q). 

Suppose first that Supp(d) = So- If u n G Aq for infinitely many n G N then 
T = So- Otherwise, since A is finite we can suppose w.l.o.g. that there exists a 
letter a G A \ Aq such that for every n G N, v n a is a prefix of u n . Let also w n 



such that u n = v n aw n . Let S± = So ■ a . Then Si ^ Sq because a $ Aq and Si 
is clearly ((-reachable from So- Moreover (w n ) Tl gN is a limit-path from Si to T, 
this completes the proof. 

Suppose now that Supp(d) ^ Sq. Let M[So, Aq] the probabilistic automaton 
obtained from A by restriction to the alphabet A 0 and to the state space So- By 
definition of A 0 , .4[So, A 0 ] is stable and it is ([-acyclic because A is. According 
to the leaf lemma, A[So, Ao] has a unique leaf. Let Si be this unique leaf. Since 
Supp(5) is limit-reachable from So in A[So,Ao], according to the leaf lemma 
again, Si C Supp(A) hence Si ^ So- Moreover, since it is the unique leaf, Si is 
((-reachable from So in M[So, A 0 ] hence in A as well. For every n € N, let w n such 
that u n = v n w n . Then (w n ) ne ^ is a limit-path from Si to T. This completes the 
proof. □ 

Proof (of Proposition 8 ). Apply again and again Lemma 4 to build a sequence 
So,Si,S 2 ,... such that for every k, Sk ^ S^+i, Sk+i is ((-reachable from Sk 
and T is limit-reachable from Sk+i- As long as Sk 7 ^ T, Lemma 4 is used to 
build Sfc+i- Since A is subset-acyclic, the sequence has length at most 2 ^ thus 
for some fc, Sk = T. Since ((-reachability is transitive, this proves that T is ((- 
reachable from Sq. □ 



